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Abstract 

A tenet of multitiered systems of support and response to intervention (MTSS-RTI) is that lack 
of response to instructional intervention is explained by classroom experiences and behaviors 
given opportunities to learn. We investigated the potential of filling this information gap in 
MTSS-RTI decision making using ecobehavioral observation to inform steps that could be taken 
for children not responding to preschool literacy instruction. Data analyses indicated that (a) 
teachers implemented a uniform pattern of daily activities providing children with infrequent 
opportunity to learn literacy, (b) the proportion of children’s co-occurring academic engagement 
also was low but varied widely depending on the activity and teacher’s literacy focus, and (c) 
children’s personal risk characteristics moderated the strength of relationships. Novel was the 
finding that in some activities and teacher behaviors, teachers appeared to be differentiating 
instruction benefiting children with individualized education programs and dual-language-learner 


risk. Implications are discussed. 
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It is challenging for preschool teachers to 
meet the needs of all the children they serve 
because of the diversity in the language and 
early literacy skills children bring to the class- 
room. Reports indicate that preschool chil- 
dren in the lower quartile of language and 
literacy skills proficiency are often those with 
limited prior experience, low socioeconomic 
status (SES), dual-language experiences, or 
developmental delays (Carta & Driscoll, 
2013; Neuman, Kaefer, & Pinkham, 2018). 
Children in the lower quartile make progress 
in a preschool year but are not likely to catch 
up to typically developing peers (Greenwood 
et al., 2013). These children remain at risk for 
not being ready for school and not learning to 
read proficiently (Hoff, 2006, 2013; Zill & 
Resnick, 2006). It has been suggested that 


response to instructional intervention predicts 
which children readily learn literacy skills and 
why others struggle (Connor et al., 2009; 
Connor, Morrison, & Petrella, 2004). The 
approach known as multitiered systems of 
support and response to intervention (MTSS- 
RTI; Division for Early Childhood of the 
Council for Exceptional Children, National 
Association for the Education of Young Chil- 
dren, & National Head Start Association, 
2014; VanDerHeyden & Snyder, 2006) 
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Figure I. Tilly’s multitiered-system-of-support universal screening and progress-monitoring model 


(Tilly, 2008). 


uniquely addresses this diversity in preschool 
children’s entry skills by letting RTI data 
inform decisions about the need for more or 
less intensive (differentiated) instruction 
(Merrell & Buchanan, 2006). 

In MTSS-RTI, universal screening and 
progress-monitoring data provide the evidence 
that practitioners need when making interven- 
tion decisions (Batsche, Castillo, Dixon, & 
Forde, 2008; Snyder, Wixson, Talapatra, 
& Roach, 2008). Significant progress has 
been made in developing the approach for 
preschool (Carta et al., 2016). For example, 
Individual Growth and Development Indicators 
(IGDIs) for infants, toddlers, and preschoolers 
provide early educators with the MTSS-RTI 
measurement they need (Greenwood, Carta, 
& McConnell, 2011; McConnell, Wackerle- 
Hollman, & Bradfield, 2014; Neuman & 
Carta, 2011). 

Although IGDIs are good at indicating 
who may benefit from more intensive instruc- 
tion (McConnell et al., 2014), they provide 


little help in determining what may be con- 
tributing to an individual child’s lack of learn- 
ing progress (Why is the learning problem 
happening?), what steps should be taken 
going forward (What should be done about 
the problem?), and whether these steps are 
being implemented as planned (Is a solution 
being implemented?). This is illustrated in 
Tilly’s (2008) problem-solving model (Figure 
1). We know of no methods that inform the 
steps to be taken after problem identification, 
particularly ones that bring greater precision 
to these questions. 

Ideally, a useful method would estimate a 
focal child’s opportunity to learn language 
and literacy as well as the child’s behavior 
co-occurring with these opportunities to 
learn. When considered together, they indi- 
cate the success or failure of instruction in 
providing the necessary experiences needed 
to promote literacy. Both variables could be 
altered by the teacher either by providing 
greater exposure to target content or by 


Greenwood et al. 


273 


embedding use of evidence-based practices 
known to promote child responding and aca- 
demic engagement. These variables could be 
operationalized as the (a) proportion of 
observed time that learning opportunities are 
provided by the teacher and (b) the likeli- 
hood of a child’s engaged response occurring 
given the instructional opportunity, an 
instruction—child behavior dependency. This 
method would enable early childhood practi- 
tioners to collect and interpret these indica- 
tors for individual children and to make 
data-based decisions about implementing 
changes using evidence-based practices 
(e.g., Ziolkowski & Goldstein, 2008) with 
implementation supports (e.g., practice- 
based coaching; Snyder, Hemmeter, & Fox, 
2015) that could be tested for function. 

Widely used preschool observational mea- 
sures currently do not provide this informa- 
tion. They are not designed to inform 
decisions about what children should be 
taught and what they should learn in pre- 
school (Shanahan & Lonigan, 2008), do not 
measure child-level information (CLASS; 
Pianta, La Paro, & Hamre, 2008), or do not 
measure instruction—child behavior co-occur- 
rence (Downer, Booren, Lima, Luckner, & 
Pianta, 2010). Those few that do measure 
child-level information and child behavior 
co-occurrences require video capture and 
coding software to be used outside of the 
classroom and are not yet feasible for practi- 
tioners (Connor et al., 2009; Pelatti, Piasta, 
Justice, & O’Connell, 2014). 

Ecobehavioral observation data (1.e., the 
Code for Interactive Recording of Children’s 
Learning Environments [CIRCLE]) are a 
potential solution to all of these problems 
(Greenwood et al., 2018) because they pro- 
vide the needed information on a child’s 
opportunity to learn and instruction—child 
behavior co-occurrence (Kontos & Keyes, 
1999; D. Powell, Burchinal, File, & Kontos, 
2008). CIRCLE is the outgrowth of ecobehav- 
ioral observational research (e.g., Greenwood, 
Carta, Kamps, Terry, & Delquadri, 1994) 
based on the premise that child behavior is 
malleable and responsive to situational varia- 
tions provided by teachers in the classroom 


environment (Fredricks, Blumenfeld, & Paris, 
2004). 

Guiding CIRCLE development was the 
concept that a child’s learning from classroom 
instruction is a function of the interaction 
between the opportunity to learn, child behav- 
ior given opportunities, and child risk charac- 
teristics (i.ec., low SES, individualized 
education program [IEP], or dual-language 
learner [DLL]). CIRCLE is designed to 
inform these considerations in intervention 
decision making. Regarding the opportunity 
to learn, we analyze the composite occurrence 
levels of teacher literacy focus (TLF; What is 
taught?) and the activity structures (Where is 
it taught?). CIRCLE’s TLF composite, for 
example, reflects the opportunity to learn pho- 
nological awareness, alphabet and print con- 
cepts, story comprehension, other 
comprehension, vocabulary, reading, and lit- 
eracy involvement provided by the teacher 
(see Table 1). Preschool teachers organize the 
day in terms of activities in which literacy 
content can be embedded in contrast to the 
subject matter content that drives elementary 
education schedules. CIRCLE activity struc- 
tures include center, story time, large and 
small groups, and individual activities (con- 
solidated as “academic activities”) and meals 
and snacks; cleanup, setup, and transition; 
personal care; therapy; restricted access; and 
none listed (consolidated as “other, nonaca- 
demic activities”) (Table 1). 

For questions concerning children’s behav- 
ior (What is the child doing?), we consolidate 
CIRCLE’s 13 child behaviors into three theo- 
retical composites (Greenwood et al., 2018). 
These are children’s academic engagement 
(CAE), other engagement, and other behavior 
(see Table 1). CAE is the sum of writing, read- 
ing words or letters out loud, academic manip- 
ulation, academic verbal response, and 
academic attention. Compared to prior reports 
that excluded academic attention in CAE 
(Greenwood, Horton, & Utley, 2002), we 
included academic attention here because pre- 
school children’s repertoire of classroom aca- 
demic skills is only just emerging (see Table 
1). Other engagement is the sum of play, sing- 
ing and music, nonacademic manipulation, 
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Table I. CIRCLE Descriptives. 


Unconditional 


Component Frequency Probability Composites 
Activity structures 
Center 6,310 30 Academic activities 
Story time 1,379 06 
Large group 6,658 31 
Small group 2,301 ai 
Individual activities 345 02 
Meals, snacks 2,050 10 Other activities 
Cleanup, setup, transition 1,937 09 
Personal care 252 Ol 
Therapy 28 .00 
Restricted access 27 .00 
None of those listed I .00 
Teacher literacy focus (TLF) 
Phonological awareness 242 Ol Yes TLF 
Alphabet/print concepts 651 .03 
Story comprehension 351 02 
Other comprehension |,227 06 
Vocabulary 195 Ol 
Reading 651 .03 
None of those listed 17,971 84 No TLF 
Child behavior 
Writing 251 Ol Academic engagement 
Reading aloud 112 01 
Academic manipulation 1,197 06 
Academic verbal response 315 01 
Academic attention 2,990 14 
Music, recitation 357 02 Other engagement 
Pretend play 152 Ol 
Nonacademic manipulation 4,443 21 
Gross motor 1,530 .07 
Eating, drinking 1,474 .07 
Nonacademic attention 872 .04 
Competing behavior 9l .00 Other behavior 
None of those listed 7,504 35 
Total 21,288 1.00 


Note. CIRCLE = Code for Interactive Recording of Children’s Learning Environments. 


gross motor, eating and drinking, and nonaca- 
demic attention. Other behavior is a compos- 
ite of inappropriate behaviors (e.g., aggression, 
noncompliance) and any other child behaviors 
not otherwise defined in CIRCLE. 

For questions about the likelihood of a pos- 
itive relationship occurring between instruc- 
tion and child behavior, we analyze the data 
preserving co-occurrence as the focal variable 


(Kontos, Burchinal, Howes, Wisseh, & Galinsky, 
2002). Because CIRCLE data are multiply time 
sampled, the probability of a child’s behavior 
B, given TLF A (an instruction—child behavior 
dependency), can be estimated. These data 
help confirm the extent to which academic 
engagement is occurring at the moments in 
which the child actually has the opportunity to 
learn and uniquely reveals the activities 
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wherein desired child behaviors are occurring 
differentially. This information is particularly 
useful in analyzing RTI and differentiating 
instructional interventions (D. Powell et al., 
2008) because it provides a window on the 
opportunity to learn and exactly how a focal 
child is responding to a given intervention. 
With this information, teachers can do more 
or less of what is working or add strategies to 
those not working to boost child response 
when lacking or inappropriate. 

Preliminary findings have supported CIR- 
CLE’s potential based on analyses of an extant 
CIRCLE data sample (Greenwood et al., 
2018). That extant sample was part of data 
collected in a 2-year wait-list randomized 
control trial of the efficacy of an enhanced lit- 
eracy intervention (Greenwood, Abbott, 
Beecher, Atwater, & Petersen, 2017). Thirty- 
nine lead teachers (20 in Year 1, 19 in Year 2) 
participated. The child sample with CIRCLE 
observations included 117 (59 in Year 1, 58 in 
Year 2) who were all 4 to 5 years old. Twenty- 
seven percent of children had an IEP in Year 
1, versus 28% in Year 2. None were DLLs. 

Key findings indicated that children in 
most classrooms prior to intervention experi- 
enced low levels of exposure to literacy- 
focused instruction (Greenwood et al., 2018). 
We also reported wide variation in the amount 
of the literacy-focused instruction teachers 
embedded during daily activities. For exam- 
ple, children were much more likely to experi- 
ence literacy instruction during story time and 
large groups, and least likely to experience 
literacy instruction during centers, small 
groups, and other activities. Other researchers 
have reported that limited opportunity maybe 
a reason for some literacy learning problems 
and that improvements could result by increas- 
ing exposures to some activities over others. 
For example, teachers could reduce time in 
centers and other activities in favor of spend- 
ing more time in large groups and story time. 
Teachers could also embed more literacy- 
focused instruction during these times 
(Downer, Rimm-Kaufman, & Pianta, 2017; 
Vitiello, Booren, Downer, & Williford, 2012). 

CAE also has been positively associated 
with language and literacy outcomes and 


readiness (Chien et al., 2010; Sabol, Bohl- 
mann, & Downer, 2018); however, the activi- 
ties preschool teachers provide are highly 
variable in promotion of CAE. D. Powell, 
Burchinal, File and Kontos (2008) reported 
that children were involved in whole-group 
settings for 52% of the observation time dur- 
ing which they were least likely to be actively 
engaged. Children were more likely engaged 
during academic activities when involved in a 
peer group and given teachers’ affirmations or 
monitoring. We also reported that the propor- 
tion of CAE was low, only .30 overall, but 
much more likely given the teacher provided a 
focus on literacy (.68) compared to nonliter- 
acy (.32) (Greenwood et al., 2018). 

Results also shed light on the challenges 
teachers face given the finding that children’s 
typical instruction—response dependencies 
were moderated by children’s literacy risk and 
IEP status. Children at greater risk due to 
weaker literacy skills or special needs (an 
IEP) had significantly weaker instruction— 
response dependencies compared to children 
without these risks in some but not all situa- 
tions. For example, children at greater literacy 
risk and children with an IEP had weaker 
story time-CAE dependencies compared to 
children with low literacy risk and without 
IEPs. These results appeared to be helpful 
information in guiding how instructional sup- 
ports may need to be differentiated (intensi- 
fied) to strengthen the likelihood of CAE for 
higher-risk children in specific activities. 

Our general thesis is that the addition of 
CIRCLE observation data to preschool 
MTSS-RTI decision making will overcome 
present limitations in logic, precision, and 
evidence. For example, CIRCLE observa- 
tion data conducted by an MTSS-RTI team 
member (consultant or coach) could be 
used to investigate, analyze, and report 
information that informs the questions, 
Why is the problem happening? What 
should be done about it? Is the solution 
being implemented? and Is it working? (see 
Figure 1). The long-term goal is to improve 
understanding of the effects of preschool 
language and literacy instruction through 
descriptive and nomothetic research, and 
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evidence from ideographic modifications 
of children’s interventions. 

The specific purpose of this research, 
however, was to replicate CIRCLE’s sensi- 
tivity to variations in classroom instruction 
and children’s co-occurring academic 
engagement as reported previously. Replica- 
tions are important because they test new 
research questions, apply new statistical 
methods, and check the reproducibility of 
earlier findings given that replication efforts 
often produce no effects or weaker effects 
than originally reported (Open Science Con- 
sortium, 2015). In addition to replicating 
Greenwood et al. (2018), we were able to 
examine the moderating effects of DLL sta- 
tus. We addressed the following questions. 


1. To what extent did teachers provide 
language- and literacy-focused instruc- 
tion (opportunities to learn) in daily 
activities? This question sought to rep- 
licate earlier findings regarding infre- 
quent literacy opportunity levels 
provided children during daily activi- 
ties and overall, when the opportunity 
to learn is one link to understanding a 
child’s lack of response to instruction. 

2. Was children’s academic engagement 
associated with momentary variations 
in activities and _ literacy-focused 
instruction? This question sought to 
replicate earlier findings regarding 
whether the probability of desired 
children’s academic behavior varied 
significantly given activities and lan- 
guage and literacy-focused instruc- 
tion—another link to understanding a 
child’s lack of response to instruction. 

3. Do children’s personal risk character- 
istics moderate  instruction—child 
behavior dependencies? This ques- 
tion sought to replicate earlier find- 
ings regarding whether personal risk 
characteristics (i.e., literacy risk and 
IEP status) strengthened or weakened 
children’s instruction—response depen- 
dencies in various situations and also 
extended this analysis to include chil- 
dren with DLL status not previously 


examined. The work is important 
because it informs the potentially 
negative function of risk factors on 
children’s response to instruction but 
also may confirm the presence of 
instructional differentiation in the 
case of children at risk. 


Method 
Background 


Both the initial report (Greenwood et al., 
2018) and the current investigation used 
extant data samples in secondary analyses. 
The sample used in this replication study was 
first reported by Greenwood et al. (2013). 
That study was a longitudinal descriptive 
comparison of the progress children in made 
in one preschool year. Language and literacy 
outcome measures were both formative and 
summative. CIRCLE proportion occurrence 
data were used as an indicator of CAE and 
TLF. Primary findings indicated that (a) all 
children in their preschool year prior to kin- 
dergarten made small academic gains, but 
children starting the year in lower Tiers 2 and 
3 performance levels did not close skills 
gaps; and (b) variations were noted by pro- 
gram types of varying sociodemographics 
(i.e., pre-K, Head Start, Title I, and tuition 
based) and in CIRCLE TLF exposure and 
CAE occurrence. 

Although the two extant samples were 
common in terms of measures used and 4- and 
5-year-old participants, there were differ- 
ences. For example, the Greenwood et al. 
(2013) sample was collected 4 years earlier 
than Greenwood et al. (2017). Sample also 
varied in numbers of sites (V = 4 vs. 2), pro- 
grams (N = 65 vs. 2), and children enrolled (V 
= 659 vs. 117), respectively. CIRCLE data 
were collected on one occasion for each child 
at midyear in Greenwood et al. (2013) versus 
multiple occasions gathered sequentially over 
2 years (Greenwood et al., 2017). The 2013 
sample included pre-K, Title I, Head Start, 
and tuition-based programs; 9% of children 
had IEPs and 23% were DLLs. The 2017 
sample included two programs (one reverse 
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Table 2. Literacy Skills Sample Characteristics. 
Get Ready to Read raw scores 

Variable M SD Skewness Kurtosis 
Overall 11.7 45 -0.017 -1.020 
Literacy risk group* 

High (25th percentile) 6.2 1.6 

Mid (26th to 75th percentile) 11.7 2.0 

Low (>75th percentile) 17.4 1.2 
IEP status? 

None 11.8 4.5 

IEP 10.2 3.7 
DLL status‘ 

None 12.4 0.3 

DLL 9.4 0.5 


Note. N = 354 children, 65 classrooms. IEP = individualized education program; DLL = dual-language learner. 


*F(63, 286) = 2.3332, p = .000I. 
*F(1, 338) = 1602.2, p = .0001. 
“F(1, 338) = 3.730, p = .05. 


inclusion vs. one state-funded pre-K); 15% 
had IEPs and there were no DLLs. 


Sample 


Programs, children, and teachers. The children 
reported by Greenwood et al. (2013) were 
recruited from programs with language and 
literacy goals and that were using an early lit- 
eracy core curriculum with an identifiable 
scope and sequence; the majority of early lit- 
eracy instruction occurred in English. For the 
current analysis, a subsample of 354 children 
with complete observation and _ literacy 
screening data participated. This subsample 
was stratified to represent classroom differ- 
ences in children’s literacy skills by randomly 
drawing six per classroom to receive observa- 
tions (see Table 2). For various reasons of 
missing data and efforts to replace data when 
children moved away early in the study, the 
number of children who were actually 
assessed in each classroom ranged from two 
to seven (see Table 2). The percentages of 
children in the three literacy risk groups were 
28%, 46%, and 26% for the high-, mid-, and 
low-risk groups, respectively (see group cut 
points below); 9% had IEPs; and 24% were 
DLLs. As a group, DLLs were more precisely 


defined as (a) Spanish-English DLLs but most 
comfortable speaking Spanish (15%), (b) 
Spanish-English DLLs but most comfortable 
speaking English or English and Spanish 
(5%), and (c) DLLs speaking languages other 
than Spanish (4%), for example, Hmong. All 
enrolled teachers and staff provided informed 
consent. Classroom lead teachers reported an 
average of 10.4 years of teaching experience, 
and only 5% were nondegreed. The reported 
degrees in early childhood were graduate 
(20%), 4 year (45%), 2 year (8%), and Child 
Development Associate (3%). Nineteen per- 
cent had degrees in other fields. 


Measurement 


Child and family characteristics. Sociodemo- 
graphic characteristics of the children and 
their families were assessed using a 25-item 
parent survey. Date of birth, age, gender, 
race-ethnicity, and disability status indicated 
by the IEP were collected for each child, 
along with the primary caregiver’s educa- 
tional attainment and languages spoken at 
home (DLL status). 


Children’s preliteracy screen. The Get Ready 
to Read (GRTR; Whitehurst & Lonigan, 
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2001b) was used (Phillips, Lonigan, & 
Wyatt, 2009; Whitehurst & Lonigan, 2001a, 
2001b). This screener is a brief, 20-item mul- 
tiple-choice measure of letter-names and let- 
ter-sounds, print knowledge, emergent 
writing, and phonological awareness with 
alpha = .78, test-retest reliability = .80, and 
criterion validity = .58 to .69 (Phillips et al., 
2009). The raw score was used in analyses. 


CIRCLE measurement. The CIRCLE data 
reported originally (Greenwood et al., 2013) 
were used to describe and quantify (a) class- 
room activities (e.g., story, small groups), 
(b) teacher’s behavior (e.g., focus of instruc- 
tion), and (c) a target child’s behavior (i.e.., 
academic engagement, other engagement) 
using a 15-s, momentary time sample 
method (Kennedy, 2005). Momentary time 
sampling is the best sampling approach for 
estimating frequency of occurrence (Ary, 
1984; J. Powell, Martindale, & Kulp, 1975). 
CIRCLE data were recorded on tablet 
devices using a data entry application 
(Atwater, Lee, Montagna, Reynolds, & 
Tapia, 2009) that paced data entry by timing 
the intervals. 

In momentary time sampling, observers 
record events simultaneously when prompted 
by an unobtrusive signal. At the signal, observ- 
ers selected the event best describing the activ- 
ity occurring at that moment from a drop-down 
entry field in the application. Teacher’s behav- 
ior and child events were recorded in a similar 
manner in the next two intervals. The activity, 
teacher, and child recording sequence was 
repeated until the end of the observation. 
Observations of the teacher lasted 90 min, dur- 
ing which time three children were sequentially 
observed for 30 min each. Children’s observa- 
tions rotated through 10-min blocks to repre- 
sent variability in the activities children were 
experiencing. The remaining three children 
were observed the next day. Observations were 
scheduled by the teacher’s report of when liter- 
acy-related activities were most likely to occur 
(e.g., centers, story time). 

A single measurement director trained, 
planned, and supervised the multisite data 
collection. This director worked with three 


cross-site coordinators who supervised and 
monitored implementation. Local coordina- 
tors were trained and certified by the CIR- 
CLE’s developer. All staff members at sites 
were trained to use the same measures prior to 
study data collection. All met calibration stan- 
dards on procedural and measurement reli- 
ability, and reliability checks were conducted 
using 20% of observations at each site. The 
reliability estimates for CIRCLE data were 
84.6% to 97.5% for exact percentage agree- 
ment overall; kappa ranged from 0.70 to .88 
per study site on individual CIRCLE codes 
(Greenwood et al., 2013). 


Statistical Analysis 


As with Greenwood et al. (2018), sample 
representativeness and distributional fea- 
tures of the data were examined using sim- 
ple descriptive analyses (i.e., frequencies, 
means and standard deviations, proportions, 
and graphical displays). Like Greenwood 
et al. (2018), results indicated that the dis- 
tribution of children’s literacy skills was 
representative of the GRTR normative sam- 
ple. The mean early literacy skills of this 
sample on the screener’ raw score was 11.7 
(SD = 4.5; see Table 1) compared to M = 
9.1 (SD = 4.3) originally reported by 
Whitehurst and Lonigan (2001b). Skewness 
of the distribution overall was near zero 
(-0.017, SE = 0.130), where asymmetry = 
0 in the normal distribution. Kurtosis indi- 
cated a degree of flatness (-1.020, SE = 
0.260), where kurtosis = 0 in the normal 
distribution. 

Risk moderators included literacy risk and 
IEP status as reported by Greenwood et al. 
(2018), as well as DLL status. Literacy risk was 
established by coding cut points on the GRTR 
score to define three groups, where high group 
= 0 Gf <25th percentile), mid group = 1 (if 
>25th percentile and <75th percentile), and 
low group = 2 (>75th percentile). IEP and DLL 
risk variables were the result of binary coding of 
teacher-provided information, where the Eng- 
lish, monolingual learner group = 0 and the 
DLL status group = 1, and where the no-IEP 
group = 0 and the IEP group = 1. Univariate 
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Table 3. Probability of Teacher Literacy Focus Given Activity Structures. 
Teacher literacy focus (TLF) 

Activity structure (AS) No Yes Total Prob (AS) 
Large group (LG) 

Prob (TLF / LG) 0.74 0.26 

Frequency 4,935 1,723 6,658 31 
Center (C) 

Prob (TLF / C) 0.96 0.04 

Frequency 6,079 231 6,310 30 
Other activities (OA) 

Prob (TLF / OA) 0.97 0.03 

Frequency 4,183 112 4,295 .20 
Small group (SG) 

Prob (TLF / SG) 0.84 0.16 

Frequency 1,941 360 2,301 Ad 
Story time (ST) 

Prob (TLF / ST) 0.37 0.63 

Frequency 507 872 1,379 06 
Individual activities (IA) 

Prob (TLF / IA) 0.94 0.06 

Frequency 326 19 345 02 
Total 

Prob (TLF) 0.84 0.16 

Frequency 17,971 3,317 21,288 1.00 


Note. Prob = probability. 


ANOVA indicated significant differences 
between the GRTR means for all three risk fac- 
tors (see Table | note). We also explored the 
pairwise overlap in these variables that indicated 
that both IEP and DLL risk were overlapping 
with literacy risk (DLL vs. literacy risk, p = 
—.40; IEP vs. literacy risk, p =—.18) but not each 
other (p = —.03) 

We analyzed the CIRCLE data, preserving 
activity, teacher, and child behavior co-occur- 
rence as the focal variable (Kontos et al., 
2002), by combining these three 15-s intervals 
into one 45-s record. The CIRCLE data set 
included 21,288 individual records. The 
momentary relationships between these events 
was indicated by conditional probabilities. The 
conventional reference to a conditional proba- 
bility is in terms of the outcome of interest 
(e.g., CAE or TLF) preceded by the expression 
“given the condition,” in this case the activity 
or TLF, or the probability of CAE / activity = 
x or probability TLF / activity = x. 


Conditional probabilities were computed 
using two-way marginal frequency tables (see 
Tables 3 and 4). The first research question, 
regarding children’s exposure to activities and 
TLF, is addressed in Table 3. The second 
research question, regarding the conditional 
probability of children’s behaviors (i.e., CAE, 
other engagement, and other behavior) given 
activities and TLF, is addressed in Table 4. 
Chi-square tests of independence were con- 
ducted to compare differences in these depen- 
dencies using alphas set to Bernoulli-corrected 
levels (see Table 5). 

To address the third research question, gener- 
alized linear mixed modeling (GLMM) was 
used. GLMM uniquely addressed features of the 
data that included lack of independence created 
by multiple observations per classroom (Kontos 
et al., 2002) and sparseness in the occurrence of 
some events and behaviors. The approach also 
enabled modeling multiple sources of depen- 
dence due to child characteristics without 
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Table 4. Conditional Probabilities of Child Behavior Given Activity Structures or Teacher Literacy 


Focus. 


Child behavior composite (CB) 


Academic Other Other 
CIRCLE context engagement engagement behavior Total Prob (AS) 
Activity structures (AS) 
Large group (LG) 
Prob (CB / LG) 0.28 0.21 0.52 
Frequency 1,856 1,347 3,455 6,658 31 
Center (C) 
Prob (CB / C) 0.20 0.60 0.20 
Frequency 1,260 3,777 1,273 6,310 30 
Other activities (OT) 
Prob (CB / OA) 0.05 0.55 0.40 
Frequency 199 2,381 1,715 4,295 .20 
Small group (SG) 
Prob (CB / SG) 0.25 0.48 0.27 
Frequency 582 1,093 626 2,301 Al 
Story time (ST) 
Prob (CB / ST) 0.60 0.09 0.31 
Frequency 829 120 430 1,379 06 
Individual activities (IA) 
Prob (CB / IA) 0.40 0.32 0.28 
Frequency 139 110 96 345 .02 
Teacher Literacy Focus (TLF) 
No 
Prob (CB / No TLF) 0.16 0.47 0.37 
Frequency 2,847 8,535 6,589 17,971 84 
Yes 
Prob (CB / Yes TLF) 0.61 0.09 0.30 
Frequency 2,018 293 1,006 3,317 16 
Total 
Prob (CB) 0.23 0.41 0.36 
Frequency 4,865 8,828 7,595 21,288 1.00 


Note. CIRCLE = Code for Interactive Recording of Children’s Learning Environments; Prob = probability. 


requiring that all events be experienced by all 
children to be included in the model. Additional 
advantages included the testing of main effects 
and interactions among predictors at any level of 
analysis, permitting the inclusion of incomplete 
child- or teacher-level data under the assumption 
of missing at random, and allowing for unbal- 
ance in the number of observations per sampling 
unit. The levels in the analysis were child records 
(N = 21,288 at Level 1), child (V = 354, intra- 
class correlation [ICC] = .237 at Level 2), and 
classroom (N = 65, ICC = .341 at Level 3) using 
binary outcomes and with pairwise comparisons. 


The first step in the analysis estimated the 
extent to which CAE was predicted by co-occur- 
ring activities or TLF at Level 1. The second 
step estimated the extent to which these relation- 
ships varied systematically by child literacy, 
DLL, and IEP risk status. Because the GRTR 
score was a continuous variable, literacy risk 
was mean-centered in these moderation models. 
The binary outcome variable in the models was 
CAE versus the composite of other engagement 
plus other behavior, or all other behavior (not 
CAE). The comparison variable in each model 
was that least associated with the occurrence of 
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Table 5. Pairwise Differences in the Probability of Child Academic Engagement Given Activity 

Structures and Teacher Literacy Focus. 

Pairwise comparison Beta SE Z p Odds ratio 

Activity structure — child academic engagement® 
Center vs. other activities 1.76 0.09 20.03 <.0l 5.83 
Story time vs. other activities 3.80 0.11 34.88 <.01 44.83 
Large group vs. other activities 2.19 0.09 25.20 <.0l 8.96 
Small group vs. other activities 2.30 0.10 22.46 <.0l 9.95 
Individual vs. other activities 2.83 0.16 17.28 <.0l 17.00 
Story time vs. center 2.04 0.09 23.65 <.0l 7.70 
Large group vs. center 0.43 0.06 7820 <0 1.54 
Small group vs. center 0.54 0.08 6.70 <.0l 1.71 
Individual vs. center 1.07 0.15 7.08 <.0l 2.92 
Large group vs. story time -16l 008 -19.64 <I 0.20 
Small group vs. story time -1.51 0.10 -1543  <.0l 0.22 
Individual vs. story time -0.97 0.16 5.96 <.0l 0.38 
Small group vs. large group 0.10 0.07 1.41 70 ltl 
Individual vs. large group 0.64 0.15 429 <.0l 1.90 
Individual vs. small group 0.54 0.16 3.40 <.0l 1.71 

Teacher literacy focus —> child academic engagement” 
Yes vs. no 2.17 0.05 42.53 <.0l 8.72 


2087, df =5,p < 01. 
3213, d'= 1,p <.01. 


bya 


CAE, against which differences with other vari- 
ables were tested. These comparison variables 
were other activities and no TLF. To assist inter- 
pretation of these comparisons, we used odds 
ratios. Odds ratios represent the odds that an out- 
come will occur given a particular exposure, 
compared to the odds of the outcome occurring 
in the absence of that exposure, or (probability 
of A occurring / probability of A not occurring) / 
(probability of B occurring / probability of B not 
occurring) (Singer & Willett, 2003, p. 388). 


Results 


To What Extent Did Teachers 
Provide Language- and Literacy- 
Focused Instruction? 


Teachers organized children’s preschool day 
with the activities reflected in the “Total” col- 
umn of Table 3. This distribution was similar 
to that reported by Greenwood et al. (2018). 
The proportion of intervals that children spent 
in daily activities were distributed as large 
group (31%), center (30%), other activities 


(20%), small group (11%), story time (6%), 
and individual activities (2%). However, the 
proportion of intervals that teachers provided 
children TLF was only 0.16, or 16% (see 
Table 3). Children were least likely to experi- 
ence TLF in the activities wherein they spent 
the majority of their daily time. The exception 
was large-group activities, where the proba- 
bility of TLF co-occurring was higher than 
most, at only 0.26. Children were most likely 
to experience TLF during story time (0.63). 


Was CAE Significantly Associated 
With Momentary Variations in 
Activities and TLF? 


Probability of CAE given activities. The probabil- 
ity of CAE varied significantly given variations 
in the activities (x? = 1067.7, df = 4, p < .01; 
see Table 4, upper panel). CAE was signifi- 
cantly more likely to occur during story time, 
individual activities, and large group compared 
to small group, center, and other activities (see 
Table 4), replicating Greenwood et al. (2018). 
Inversely, children’s other engagement and 
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other behavior were significantly more likely 
to occur in small group, center, and other activ- 
ities. Pairwise comparisons indicated that CAE 
occurred significantly more often in all five 
activities when compared to other activities 
and to centers. CAE was more likely to occur 
during story time compared to large and small 
groups and individual activities. The probabil- 
ity of CAE between small and large groups was 
not significantly different (see Table 5). 


Probability of CAE given TLF. The probability of 
CAE varied significantly depending on varia- 
tion in TLF (x’ = 1461.5, df= 1,p < .01; see 
Table 4, lower panel, and Table 5, lower 
panel). CAE was significantly more likely to 
occur when the teacher was focused on liter- 
acy compared to no TLF. The probability of 
academic engagement occurring was highest 
(0.61) given TLF compared to only 0.16 given 
no TLF (see Table 4). Children’s other engage- 
ment and other behavior were significantly 
more likely given no TLF (see Table 4). 


Did Children’s Risk Characteristics 
Moderate Instruction—Behavior 
Dependencies? 


Personal risk characteristics did moderate 
instruction—behavior dependencies  differen- 
tially. Children in the three risk groups (literacy, 
DLL, and IEP) were divergent in GRTR liter- 
acy skills (Table 1), replicating Greenwood 
et al. (2018). Children in the high-literacy-risk 
group received less TLF and CAE overall 
(Table 6). Literacy risk status was the only risk 
factor that moderated conditional probabilities 
negatively, as in Greenwood et al. (2018). For 
example, children at high literacy risk were 1.0 
times more likely to not be academically 
engaged given TLF (B = .04, p < .01) than 
children not at risk. The odds of children in the 
low-risk group being academically engaged 
during individual activities (8 = 0.14, p < 
.01) were 1.2 times lower than those of children 
not at risk. Surprisingly, IEP moderated condi- 
tional probabilities in specific activities in a 
positive, not negative, direction as reported by 
Greenwood et al. (2018). For example, the odds 
of children with an IEP being engaged during 


center (B = 0.57, p < .01) were 1.8 times higher 
than those of children without an IEP, suggest- 
ing differentiation. 

DLL risk group membership acted to mod- 
erate CAE in the positive direction. DLL risk 
groups were significantly divergent in CIR- 
CLE TLF overall, receiving 6.8% more TLF 
(M = 20.7%, SD = 16.1) than children who 
were monolingual English speaking (MV = 
13.9%, SD = 14.6), FC, 339) = 12.673, p = 
.0001. Teachers were providing significantly 
more opportunities for DLL children to learn 
phonological awareness, story comprehen- 
sion, other comprehension, and vocabulary 
(four of six CIRCLE TLF behaviors) com- 
pared to monolingual children (Table 6). 

DLL risk group membership also produced 
13.4% more CAE (M = 29.0%, SD = 19.3) 
versus monolingual English speakers overall 
(M = 15.6%, SD = 15.3), F(1, 399) = 14.556, 
p = .0001. DLL children were significantly 
more engaged in academic verbal response 
and academic attention compared to monolin- 
gual English speakers (see Table 6). In terms 
of specific activities, the odds of DLL chil- 
dren being academically engaged were 2.6 
times higher in large-group (8B = 0.94, p < 
.O1) and 3.0 times higher in small-group 
instruction (B = 1.09, p < .01) than those of 
mono-English speakers. Because the DLL 
Status < Site interaction in analyses was not 
significant, differentiation favoring DLL chil- 
dren appeared to be occurring. 


Discussion 


Improving precision in MTSS-RTI decision 
making for children not responding to lan- 
guage and literacy instruction is a need in the 
field. To make progress on this issue, the 
purpose of this study was to replicate earlier 
findings related to CIRCLE’s sensitivity to 
variations in classroom instruction and chil- 
dren’s co-occurring academic engagement. A 
second purpose was examination of old and 
new issues in the moderation of CIRCLE’s 
instruction—child behavior dependencies by 
children’s personal risk characteristics. 
Current results replicated much of the first 
report (Greenwood et al., 2018), including 
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Table 6. Differentiation of CIRCLE Teacher Literacy Focus and Child Academic Engagement for DLL 


Versus Mono-English Learners. 


MANOVA ANOVA DLL Mono-English 
Variable Wilks d F df p MS F df p M SD n M_ SD n_ Diff 
Teacher literacy focus 
DLL 0.922 4.712 6,333 .0001 
PA 143.628 16.618 1,338 0001 23 4.3 83 08 23 257 I.5 
AK 0.711 0.023 1,338 879 2.9 43 83 3.0 5.9 257 O.1 
S-Com 62.732 4.861 1,338 .028 2.3 3.9 83 1.3 3.5 257 1.0 
O-Com 580.515 7.425 1,338 .007 82 88 83 5.1 88 257 3.0 
Voc 87.94 842 1,338 .004 1.8 45 83 06 2.7 257 1.2 
Reading 0.195 0.004 1,338 .948 3.1 7.0 83 3.1 67 257 O.1 
Child academic engagement 
DLL 0.395 102.398 5,334 .0001 
Writing 14.81 1.529 1,338 217 08 2.1 83 1.3 3.4 257 -0.5 
RA 3.248 0.801 1,338 371 07 1.9 83 05 2.0 257 02 
AM 39.852 0.47 1,338 493 61 86 83 5.3 94 257 08 
AV 81.849 11.224 1,338 001 2.3 3.7 83 1.2 23 257 Il 
AA 2911.434 16.362 1,338 .0001 19.2 15.7 83 124 12.5 257 68 


Note. AA = academic attention; AK = alphabet knowledge; AM = academic manipulation; AV = academic vocalization; CIRCLE = 
Code for Interactive Recording of Children’s Learning Environments; DLL = dual-language learner; O-Com = other comprehension; 
PA = phonological awareness; RA = reading aloud; S-Com = story comprehension; VOC = vocabulary. 


the relatively uniform pattern of preschool 
activities that provided children with rela- 
tively infrequent exposure to literacy- 
focused instruction (Question 1) and similar 
variations in instruction—CAE dependencies 
given specific activities (Question 2). Results 
also provided important new findings related 
to the moderation of instruction—CAE depen- 
dencies by children’s personal risk character- 
istics (Question 3). 

Children experienced center, large group, 
small group, story time, and individual 
activities daily at nearly the same propor- 
tions as in the first report, repeating what 
appears to be a uniform organization of daily 
activities by preschool teachers. However, 
in this sample, teachers provided even lower 
levels of opportunity to learn language and 
literacy (TLF) during activities, 16%, com- 
pared to 29% and 31% a year later (Green- 
wood et al., 2018). 

As in the first report, CAE varied widely by 
activity structure (i.e., story time, large-group 
instruction) and TLF. Children spent the major- 
ity of observed time in activities that were least 
promoting of CAE (1.e., center and other activ- 
ities). CAE was most likely occurring during 
story time compared to large and small groups, 
as in the original study. The overall proportion 


of children’s CAE during observations was 
lower in this sample (23%), as compared to 
30% and 30% over 2 years in the first sample 
(Greenwood et al., 2018). 

Children’s characteristics did moderate 
(strengthen or weaken) instruction—child 
behavior dependencies. The finding that chil- 
dren with literacy risk had weaker instruc- 
tion—child behavior dependencies than 
no-literacy-risk peers was replicated. How- 
ever, new findings emerged for IEP and DLL 
risk. Literacy risk was a negative moderator 
of instruction—CAE dependency in most 
cases; however, the dependency between indi- 
vidual activities and CAE was actually 
strengthened for high-literacy-risk compared 
to low-literacy-risk children. In this context, 
findings suggested that teachers were differ- 
entiating instruction by providing instruction 
of greater intensity to these higher-literacy- 
risk children. This was a novel finding because 
we had reported the opposite finding in 
Greenwood et al. (2018). Differentiation was 
also suggested for children with IEPs, in that 
the dependency between center activity and 
CAE was strengthened compared to these 
children without an IEP. It appeared that 
teachers were providing greater instructional 
intensity to children most in need. 
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Positive moderation affects were also par- 
ticularly clear with respect to DLL status. In 
this sample, DLLs were experiencing signifi- 
cantly more TLF and CAE overall. Additional 
analyses of the CIRCLE data demonstrated 
that teachers were addressing DLL children 
with comparatively more talk about names of 
things (vocabulary), letters and sounds in 
words (phonological awareness), asking ques- 
tions about happenings or events in a story 
(story comprehension), and talking about con- 
cepts such as categories and comparisons 
(comprehension). In these contexts, DLL chil- 
dren were significantly more likely to be talk- 
ing (verbalizing) and attending to the teacher 
than mono-English peers. This greater CAE 
was most evident in the large- and small- 
group activities, where DLLs were signifi- 
cantly more likely to be academically engaged 
compared to their monolingual peers. Data 
like these were important because they 
described teachers who were differentiating 
instruction in specific situations. 


Strengths of the Study 


The findings are important to the field because 
they demonstrate the unique benefits of a 
measure, like CIRCLE, that is aligned with 
the content and standards of preschool lan- 
guage and literacy instruction and also sensi- 
tive to momentary instruction—child response 
dependencies (Greenwood et al., 2018). The 
work brings to the fore ways in which mal- 
leable environmental factors, including 
teacher behavior, affected the likelihood of 
child behaviors occurring that are known to be 
predictive of or required for early learning and 
achievement. The earlier work (Greenwood 
et al., 2018) and this replication together pro- 
vide a strong basis for descriptive, nomo- 
thetic, and idiographic modifications of 
intervention goals of preschool MTSS-RTI 
language and literacy instruction. 

Regarding description, the findings con- 
firm a relatively uniform organization of 
the preschool classroom experience across 
settings and teachers and classrooms by 
activities in which opportunities for lan- 
guage and literacy learning are embedded 


but to a variable degree. CIRCLE descrip- 
tion has provided greater understanding 
that language and literacy learning oppor- 
tunities are not yet the major time focus of 
the preschool experience, even in programs 
with language and literacy outcome goals. 
Regarding the nomothetic understanding of 
the preschool classroom experiences of 
children, this evidence provides the field 
with progress toward comparatives (bench- 
marks) for the opportunity to learn lan- 
guage and literacy in preschool as well as 
the likely co-occurrence of children’s aca- 
demic engagement overall and by different 
activities and teacher behaviors. These 
comparatives are now based on two sam- 
ples of children representative of GRTR lit- 
eracy normative outcomes. In MTSS-RTI, 
benchmarks can be used in a number of 
ways. 

Programs interested in increasing chil- 
dren’s language and literacy outcomes can use 
CIRCLE benchmarks as baseline goals to 
note the need for improvement, set reasonable 
improvement targets, and confirm the extent 
to which improvement has been achieved 
given, for example, a change in curriculum or 
professional development. CIRCLE data also 
can be used as benchmarks for ideographic 
decision making as described earlier, given a 
child’s unique profile of opportunity to learn 
and co-occurring behavior, either CAE or 
other competing behavior. CIRCLE data may 
suggest the need for providing greater TLF 
during several daily activities. Interpretation 
of CIRCLE data may suggest the need for the 
teacher to provide more frequent prompting 
of child response, followed by expansions and 
positive reinforcement. 

CIRCLE data also appear to be helpful in 
interpreting results of universal screening and 
progress monitoring (IGDI) data outcomes 
consistent with Tilly’s MTSS-RTI decision 
making model. 

The work represents an innovation in our 
capacity to understand preschool instruction 
as it is actually implemented and in guiding 
real-time changes in instruction based on evi- 
dence, compared to the less helpful broad 
analyses provided by widely used preschools 
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measures (i.e., CLASS). CIRCLE produced 
information needed to inform questions of 
why there might be an instructional explana- 
tion for a lack of progress, what might be done 
about it, and whether a planned change is 
actually being implemented. Because findings 
in both CIRCLE studies were based on inde- 
pendent, time-displaced samples representa- 
tive of the GRTR’s literacy normative 
distribution, the reliability of CIRCLE infer- 
ences about this population of children was 
strengthened. This replication also adds to the 
trustworthiness of CIRCLE findings by 
extending knowledge from the smaller sample 
(4,729 records) to this larger sample (21,288 
records). 


Limitations 


Several limitations were noted. Although 
representing a major advance in theory and 
use of precision information based on 
instruction—child behavior dependency, the 
data were constrained by use of momentary 
time sampling. This was because of its equal 
treatment of slower- versus faster-changing 
events in each 45-s co-occurrence record. 
An alternative point sampling of faster- 
changing variables nested within slower- 
changing variables under momentary 
sampling could be an improvement, as could 
real-time event recording. An axiom of 
observational measurement is that as the 
observer’s workload increases in number of 
events tracked and rates of change (com- 
plexity), reliability suffers (Dorsey, Nelson, 
& Hayes, 1986). The large number of vari- 
ables that CIRCLE can track reliably using 
momentary time sampling may not be pos- 
sible in live, real-time sampling because 
observers need to track all variables simulta- 
neously. We argue that it is better to use 
CIRCLE with this limitation because its 
comparative advantages provide actionable 
information compared to current alterna- 
tives. The advantages of point sampling and 
real-time recording may be better suited for 
coding videotaped sessions outside the 
classroom, where researchers may code, 
play back, and recode behaviors. 


We acknowledge that findings were cor- 
relational-descriptive, with data capable of 
only providing hypotheses for causality 
testing in experimental studies. However, 
such information can be appropriately used 
in MTSS-RTI as evidence needed in formu- 
lating hypotheses worth testing for function 
with individual children (e.g., Daly, Witt, 
Martens, & Dool, 1997). Children in this 
sample were observed once at midyear, and 
stability analyses between multiple fall 
occasions as was reported by Greenwood 
et al. (2018) were not possible. 


Implications for Research 


Future experimental research is needed to 
demonstrate that using CIRCLE in MTSS- 
RTI decision making actually leads to better 
progress in achieving literacy outcomes. 
Researchers can profitably use single-case 
experimental designs to test whether lan- 
guage and literacy progress-monitoring data 
pick up improvements resulting from CIR- 
CLE-based changes in instruction. Research 
is needed that focuses on the effects of pre- 
school instructional strategies and profes- 
sional development that promote greater 
intentionality by embedding academic con- 
tent and focus on literacy within and across 
daily activities (Justice & Kaderavek, 2004). 
Research also is needed separating CIR- 
CLE’s instructional constructs from other 
data-informed guidance focused on what a 
child knows and needs to learn next (Al 
Otaiba et al., 2011). Research on the effects 
of strategies that differentiate instruction to 
intensify CAE for children with greater lit- 
eracy risk, with IEPs, and who are DLLs is 
needed. CIRCLE data can be used to con- 
firm the need for differentiation as well as 
confirm that such differentiation is being 
implemented. 

For children with literacy risk, differen- 
tiation can include increasing opportunities 
to learn literacy skills. Reports indicate that 
these children struggle to learn phonemic 
awareness skills and benefit from instruc- 
tion targeting these skills (Shanahan & 
Lonigan, 2008). These children could benefit 
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from Tier 2 interventions, such as small- 
group phonemic awareness instruction 
(Goldstein et al., 2017). For children with 
IEPs, particularly those with speech, lan- 
guage, and literacy delays, this research is 
needed. Reports indicate that these children 
struggle to learn vocabulary skills but bene- 
fit from instructional support targeting pho- 
nological awareness and alphabet knowledge 
(Greenwood et al., 2013). These children 
could benefit from interventions such as 
Tier 2 small-group vocabulary instruction 
using automated storybook reading (Gold- 
stein et al., 2016) or Tier 3 individualized 
literacy skill interventions (Kaminski, Pow- 
ell-Smith, Hommel, McMahon, & Bravo- 
Aguayo, 2015). Differentiation for children 
can also include embedding oral English and 
vocabulary (Castro, Espinosa, & Paez, 2011) 
teaching strategies for DLLs given their lan- 
guage skill level (Goldenberg, 2008). 

The issue of ecological validity will 
always be a consideration in planning the 
coverage of observations. The current CIR- 
CLE data were based on 90-min observa- 
tions of the teacher and 30-min observations 
of three identified children wherein obser- 
vations occurred during literacy prime 
times, identified by teachers, where 90 min 
equates to 40% of half-day preschool pro- 
grams. CIRCLE data did confirm that 80% 
of all observed intervals occurred during 
academic activities (Table 2), with 20% 
reflecting meals and snacks, cleanup and 
transition, and child personal care. Second, 
children’s 30-min observations were cycled 
through 10-min segments to better represent 
associated activities and TLF. This duration 
of observation should not be considered a 
limitation when longer and more frequent 
observations of individual children can be 
conducted. Research also is needed that 
focuses on improving the trade-off between 
precision and complexity. Can the CIRCLE 
taxonomy of events and behaviors be imple- 
mented live in the classroom using observa- 
tional recording procedures that improve 
precision in the measurement of instruc- 
tion—child behavior dependencies while 
maintaining reliability? 


Implications for Practice 


This report joins others confirming the need to 
vastly increase instructional support for lan- 
guage and literacy learning in preschool pro- 
grams (Greenwood et al., 2013; Neuman & 
Dywer, 2009). Because measurable relation- 
ships exist between alterable instruction and 
child behavior, at least two questions now can 
be asked and answered by practitioners using 
CIRCLE: Are teachers allocating sufficient 
literacy focus to children more likely to 
respond or to children at risk? and Can we 
affect children’s learning trajectories on prog- 
ress measures by changing instructional 
arrangements? These questions are self- 
reflective for practitioners, and CIRCLE’s 
data appear to support the path to solutions 
(see Figure 1). 

Practitioners, including preschool teachers, 
early childhood special educators, and school 
psychologists interested in using evidence- 
based practices, should be encouraged to con- 
sider including CIRCLE data. Working with 
individual teachers, CIRCLE can be used to 
reveal the instructional interactions between 
teachers and children that really matter. Some 
routine preschool practices (e.g., story time) 
appear to naturally result in greater CAE and 
perhaps could be used more frequently and 
longer during the day. As with other MTSS- 
RTI measurement tools, comparative bench- 
marks will be helpful in deciding whether cur- 
rent performance is within the range needed 
for progress or if change is needed. CIRCLE 
benchmarks as criteria for TLF and CAE deci- 
sion making based on large, diverse samples 
are needed. 

For example, having several story time 
sessions or large- and small-group struc- 
tured activities distributed throughout the 
day for approximately 15 min per session 
for preschool-age children could achieve 
this goal. Other practices (e.g., center time, 
other activities) that are typically longer 
duration but least promoting of CAE could 
be replaced with activities with better 
instruction—behavior dependencies — or 
strengthened by embedding more effective 
strategies (Ziolkowski & Goldstein, 2008). 
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These findings appear to point to a means of 
improving the outcomes of preschool chil- 
dren through  greater-precision-targeted 
instructional experiences. Future research 
and practice evidence are needed to demon- 
strate whether this is the case. 


References 


Al Otaiba, S., Connor, C. M., Folsom, J. S., 
Greulich, L., Meadow, J., & Li, Z. (2011). 
Assessment data—informed guidance to indi- 
vidualize kindergarten reading instruction: 
Findings from a cluster-randomized control 
field trial. Elementary School Journal, 111, 
535-560. doi:10.1086/65903 1 

Ary, D. (1984). Mathematical explanation of error 
in duration recording using partial interval, 
whole interval, and momentary time sampling. 
Behavioral Assessment, 6, 221-228. 

Atwater, J. B., Lee, Y., Montagna, D., Reynolds, L. 
H., & Tapia, Y. (2009). Classroom CIRCLE: 
Classroom Code for Interactive Recording of 
Children’s Learning Environments (Version 
2.0). Kansas City: Juniper Gardens Children’s 
Project, University of Kansas. 

Batsche, G. M., Castillo, J. M., Dixon, D. N., & 
Forde, S. (2008). Best practices in linking 
assessment to intervention. In A. Thomas & 
J. Grimes (Eds.), Best practices in school psy- 
chology V (Vol. 2, pp. 177-194). Washington, 
DC: National Association of School 
Psychologists. 

Carta, J. J., & Driscoll, C. (2013). Early literacy 
intervention for children with special needs. In 
T. Shanahan & C. Lonigan (Eds.), Early child- 
hood literacy: The National Early Literacy 
Panel and beyond (pp. 233-254). Baltimore, 
MD: Brookes. 

Carta, J. J., Greenwood, C. R., Goldstein, H., 
McConnell, S. R., Kaminski, R., Bradfield, T. 
A.,... Atwater, J. (2016). Advances in multi- 
tiered systems of support for prekindergarten 
children: Lessons learned from 5 years of 
research and development from the Center for 
Response to Intervention in Early Childhood. 
In S. R. Jimerson, M. K. Burns, & A. M. 
VanDerHeyden (Eds.), Handbook of response 
to intervention (pp. 587-606). New York, NY: 
Springer Science+Business Media. 

Castro, D. C., Espinosa, L. M., & Paez, M. M. 
(2011). Defining and measuring quality in 
early childhood practices that promote dual 
language learners’ development and learning. 


In M Zaslow, I. Martinez-Beck, K. Tout, & 
T. Halle (Eds.), Quality measurement in early 
childhood settings (pp. 257-280). Baltimore, 
MD: Brookes. 

Chien, N. C., Howes, C., Burchinal, M., Pianta, 
R., Ritchie, S., Bryant, D. M., . . . Barbarin, 
O. A. (2010). Children’s classroom engage- 
ment and school readiness gains in prekinder- 
garten. Child Development, 81, 1534-1549. 
doi:10.1111/).1467-8624.2010.01490.x 

Connor, C. M., Morrison, F. J., & Petrella, J. N. 
(2004). Effective reading comprehension 
instruction: Examining child by instruc- 
tion interactions. Journal of Educational 
Psychology, 96, 682-698. doi:10.1037/0022- 
0663.96.4.682 

Connor, C. M., Piasta, S. B., Fishman, B., 
Glasney, S., Schatschneider, C., Crowe, E., 
... Morrison, F. J. (2009). Individualizing stu- 
dent instruction precisely: Effects of Child x 
Instruction interactions on first graders’ liter- 
acy development. Child Development, 80, 77— 
100. doi:10.1111/).1467-8624.2008.01247.x 

Daly, E. J., IU, Witt, J. C., Martens, B. K., & Dool, 
E. J. (1997). A model for conducting a func- 
tional analysis of academic performance prob- 
lems. School Psychology Review, 26, 554-574. 
doi:10.1111/).1467-8624.2008.01247.x 

Dorsey, B. L., Nelson, R., & Hayes, S. C. (1986). 
The effects of code complexity and of 
behavioral frequency on observer accuracy 
and interobserver agreement. Behavioral 
Assessment, 8, 349-363. 

Downer, J. T., Booren, L. M., Lima, O. K., Luckner, A. 
E., & Pianta, R. C. (2010). The Individualized 
Classroom Assessment Scoring System 
(inCLASS): Preliminary reliability and valid- 
ity of a system for observing preschoolers’ 
competence in classroom interactions. Early 
Childhood Research Quarterly, 25(1), 1-16. 
doi:10.1016/j.ecresq.2009.08.004 

Downer, J. T., Rimm-Kaufman, S., & Pianta, R. 
(2007). How do classroom conditions and 
children’s risk for school problems contribute 
to children’s engagement in learning? School 
Psychology Review, 36, 413-433. 

Division for Early Childhood of the Council for 
Exceptional Children, National Association 
for the Education of Young Children, & 
National Head Start Association. (2014). 
Frameworks for response to intervention in 
early childhood: Description and implica- 
tions. Communication Disorders Quarterly, 
35, 108-119. doi:10.1177/1525740113514111 


288 


Exceptional Children 85(3) 


Fredricks, J. A., Blumenfeld, P. C., & Paris, A. 
H. (2004). School engagement: Potential of 
the concept, state of the evidence. Review 
of Educational Research, 74, 59-109. 
doi:10.3102/00346543074001059 

Goldenberg, C. (2008). Teaching English language 
learners: What the research does—and does 
not—say. American Educator, 32, 42-44. 

Goldstein, H., Kelley, E. S., Greenwood, C. R., 
McCune, L., Carta, J. J., Atwater, J... . 
Spencer, T. (2016). Embedded instruction 
improved vocabulary learning using auto- 
mated storybook reading among high risk 
prescholers. Journal of Speech, Language, 
and Hearing Research, 59, 484-500. 
doi:10.1044/2015_JSLHR-L-15-0227 

Goldstein, H., Olszewski, A., Haring, C., 
Greenwood, C. R., McCune, L., Carta, J., . . 
. Kelley, E. S. (2017). Efficacy of a supple- 
mental phonemic awareness curriculum to 
instruct preschoolers with delays in early 
literacy development. Journal of Speech, 
Language and Hearing Research, 60, 89-103. 
doi:10.1044/2016_JSLHR-L-15-0451 

Greenwood, C.R., Abbott, M., Beecher, C., Atwater, 
J., & Petersen, S. (2017). Development, 
validation, and evaluation of literacy 3D: A 
package supporting Tier 1 preschool literacy 
instruction implementation and intervention. 
Topics in Early Childhood Special Education, 
37, 29-41. doi:10.1177/0271121416652103 

Greenwood, C. R., Beecher, C., Atwater, J., 
Petersen, S., Schiefelbusch, J., & Irvin, 
D. (2018). An eco-behavioral analysis of 
child academic engagement: Implications 
for preschool children not responding to 
instructional intervention. Topics in Early 
Childhood Special Education, 37, 219-233. 
doi:10.1177/0271121417741968 

Greenwood, C. R., Carta, J. J., Atwater, J., 
Goldstein, H., Kaminski, R., & McConnell, S. 
R. (2013). Is a response to intervention (RTD 
approach to preschool language and early 
literacy instruction needed? Topics in Early 
Childhood Special Education, 33, 48-64. 
doi:10.1177/0271121412455438 

Greenwood, C. R., Carta, J. J., Kamps, D., Terry, 
B., & Delquadri, J. (1994). Development 
and validation of standard classroom obser- 
vation systems for school practitioners: 
Ecobehavioral Assessment Systems Software 
(EBASS). Exceptional Children, 61, 197-210. 

Greenwood, C. R., Carta, J. J., & McConnell, 
S. (2011). Advances in measurement for 


universal screening and individual prog- 
ress monitoring of young children. Journal 
of Early Intervention, 33, 254-267. 
doi:10.1177/1053815111428467 

Greenwood, C. R., Horton, B. T., & Utley, C. A. 
(2002). Academic engagement: Current per- 
spectives on research and practice. School 
Psychology Review, 31, 328-349. 

Hoff, E. (2006). How social contexts support and 
shape language development. Developmental 
Review, 26, 55-88. doi:10.1016/j.dr.2005.11.002 

Hoff, E. (2013). Interpreting the early language 
trajectories of children from low SES and 
language minority homes: Implications for 
closing the achievement gap. Developmental 
Psychology, 49(4), 4-14. doi:10.1037/ 
a0027238 

Justice, L. M., & Kaderavek, J. N. (2004). 
Embedded-explicit emergent literacy interven- 
tion I: Background and description of approach. 
Language, Speech, and Hearing Services in 
Schools, 35, 201-211. doi:10.1044/0161- 
1461(2004/020) 

Kaminski, R., Powell-Smith, K. A., Hommel, 
A., McMahon, R., & Bravo-Aguayo, K. 
(2015). Development of a Tier 3 curricu- 
lum to teach early literacy skills. Journal 
of Early Intervention, 36, 313-332. 
doi:10.1177/1053815115581210 

Kennedy, C. H. (2005). Single case designs for 
educational research. Boston: Pearson. 

Kontos, S., Burchinal, M., Howes, C., Wisseh, 
S., & Galinsky, E. (2002). An eco-behav- 
ioral approach to examining the contextual 
effects of early childhood classrooms. Early 
Childhood Research Quarterly, 17, 239-258. 
doi:10.1016/S0885-2006(02)00147-3 

Kontos, S., & Keyes, L. (1999). An ecobehavioral 
analysis of early childhood classrooms. Early 
Childhood Research Quarterly, 14, 35-50. 
doi:10.1016/S0885-2006(99)80003-9 

McConnell, S. R., Wackerle-Hollman, A., & 
Bradfield, T. A. (2014). Universal screen- 
ing in educational settings. In R. Kettler, T. 
Glover, C. Albers, & K. A. Feeney-Kettler 
(Eds.), Early childhood literacy screening: 
Identification, implications, and interpretation 
(pp. 141-170). Washington, DC: American 
Psychological Association. 

Merrell, K. W., & Buchanan, R. (2006). 
Intervention selection in school-based 
practice: Using public health models to 
enhance systems capacity of schools. School 
Psychology Review, 35, 167-180. 


Greenwood et al. 


289 


Neuman, S. B., & Carta, J. (2011). Advancing the 
measurement of quality for early childhood 
programs that support early language and 
literacy outcomes. In M. Zaslow, T. Halle, 
& I. Martinez-Beck (Eds.), Measuring qual- 
ity in early childhood settings (pp. 51-76). 
Baltimore, MD: Brookes. 

Neuman, S. B., & Dwyer, J. (2009). Missing 
in action: Vocabulary instruction in pre-K. 
Reading Teacher, 62, 384-392. doi:10.1598/ 
RT.62.5.2 

Neuman, S. B., Kaefer, T., & Pinkham, A. M. 
(2018). A double dose of disadvantage: 
Language experiences for low-income children 
in home and school. Journal of Educational 
Psychology, 110, 102-118. doi:10.1037/ 
edu0000201 

Open Science Consortium. (2015). Estimating 
the reproducibility of psychological science. 
Science, 349, 1-8. doi:10.1126/science.aac4716 

Pelatti, C. Y., Piasta, S. B., Justice, L. M., & 
O’Connell, A. (2014). Language- and literacy- 
learning opportunities in early childhood class- 
rooms: Children’s typical experiences and 
within-classroom variability. Early Childhood 
Research Quarterly, 294), 445-456. doi:10.1016/ 
j.ecresq.2014.05.004 

Phillips, B. M., Lonigan, C. J., & Wyatt, M. A. 
(2009). Predictive validity of the Get Ready 
to Read! screener: Concurrent and long- 
term relations with reading-related skills. 
Journal of Learning Disabilities, 42, 133-147. 
doi:10.1177/00222 19408326209 

Pianta, R., La Paro, K. M., & Hamre, B. K. (2008). 
Classroom Assessment Scoring System (CLASS): 
Manual Pre-K. Baltimore, MD: Brookes. 

Powell, D. R., Burchinal, M., File, N., & Kontos, 
S. (2008). An eco-behavioral analysis of chil- 
dren’s engagement in urban public school pre- 
school classrooms. Early Childhood Research 
Quarterly, 23, 108-123.  doi:10.1016/j. 
ecresq.2007.04.001 

Powell, J., Martindale, A., & Kulp, S. (1975). 
An evaluation of time-sampling measures 
of behavior. Journal of Applied Behavior 
Analysis, 8, 463-470. 

Sabol, T. J., Bohlmann, N. L., & Downer, J. T. 
(2018). Low-income ethnically diverse chil- 
dren’s engagement as a predictor of school 
readiness above preschool classroom quality. 
Child Development, 89, 556-576. doi:10.1111/ 
cdev. 12832 

Shanahan, T., & Lonigan, C. J. (2008). Developing 
early literacy: Report of the National Early 


Literacy Panel. Retrieved from http://www. 
nifl.gov/publications/pdf/NELPReport09.pdf 

Singer, J. D., & Willett, J. B. (2003). Applied lon- 
gitudinal data analysis. Oxford, UK: Oxford 
University Press. 

Snyder, P., Hemmeter, M. L., & Fox, L. (2015). 
Supporting implementation of evidence-based 
practices through practice-based coaching. 
Topics in Early Childhood Special Education 
35, 133-143. doi:10.1177/027112141559492 

Snyder, P., Wixson, C. S., Talapatra, D., & Roach, 
A. T. (2008). Assessment in early child- 
hood: Instruction-focused strategies to sup- 
port response-to-intervention frameworks. 
Assessment for Effective Intervention, 34, 25— 
34. doi:10.1177/1534508408314112 

Tilly, W. D. (2008). The evolution of school psychol- 
ogy to a science-based practice: Problem solv- 
ing and the three-tiered model. In A. Thomas & 
J. Grimes (Eds.), Best practices in school psy- 
chology V (Vol. 1, pp. 17-36). Washington, DC: 
National Association of School Psychologists. 

VanDerHeyden, A. M., & Snyder, P. (2006). 
Integrating frameworks from early childhood 
intervention and school psychology to accel- 
erate growth for all young children. School 
Psychology Review, 35, 519-534. 

Vitiello, V. E., Booren, L. M., Downer, J. T., & 
Williford, A. P. (2012). Variation in children’s 
classroom engagement throughout a day in pre- 
school: Relations to classroom and child fac- 
tors. Early Childhood Research Quarterly, 27, 
210-220. doi:10.1016/j.ecresq.2011.08.005 

Whitehurst, G. J., & Lonigan, C. J. (2001a). 
Emergent literacy: Development from pre- 
readers to readers. In S. B. Neuman & D. 
K. Dickinson (Eds.), Handbook of early lit- 
eracy research (pp. 11-29). New York, NY: 
Guilford Press. 

Whitehurst, G. J., & Lonigan, C. J. (2001b). Get 
ready to read. Columbus, OH: Pearson. 

Zill, N., & Resnick, G. (2006). Emergent lit- 
eracy of low-income children in Head Start: 
Relationships with child and family character- 
istics, program factors, and classroom quality. 
In D. K. Dickenson & S. B. Neuman (Eds.), 
Handbook of early literacy research (Vol. 2, 
pp. 347-371). New York, NY: Guilford Press. 

Ziolkowski, R. A., & Goldstein, H. (2008). Effects 
of embedded phonological awareness inter- 
vention during repeated book reading on 
preschool children with language delays. 
Journal of Early Intervention, 31, 67-90. 
doi:10.1177/1053815108324808 


290 


Exceptional Children 85(3) 


ORCID iD 


Charles R. Greenwood © https://orcid.org/0000- 
0002-6274-3075 


Authors’ Note 


The work contained herein is based on a secondary 
analysis of a multisite data set originally reported by 
Greenwood el al. (2013). We want to thank the 
researchers and students who collected the original 
data: in Kansas, Maura Linas, Gabriela Guerrero, 
Danny Hajovsky, Joo Young Hong, Waylon How- 
ard, Na Young Kong, Tracy McElhattan, Alyssa 
Pengra, and Jennifer Robertson; in Minnesota, Scott 
McConnell, Alisha Wackerle-Hollman, Tracy Brad- 
field, Michael Rodriguez, Anthony Albano, Amanda 
Besner, Kate Clayton, Laura Potter, Megan Rodri- 
guez, and Braden Schmitt; in Ohio, Howard Gold- 
stein, Sarah Downing Sean Noe, Gin Pee, Naomi 
Schneider, Elizabeth Spencer, and Rhonda Tabbah; 
and in Oregon, Ruth Kaminski, Anne Hommel, 
Rose McMahon, and Anna Speece. Thanks are 
owed to Shye Reynolds for developing and main- 
taining the CIRCLE observation software and to Dr. 
Lesa Hoffman for her advice on the statistical 


analyses. The authors also are indebted to the par- 
ticipating children, families, and preschools in Kan- 
sas, Minnesota, Ohio, and Oregon. 

Collection of the original data set was supported by 
Grant R324C080011 from the National Center for 
Special Education Research, Institute of Education 
Sciences, U.S. Department of Education, to the 
University of Kansas in support of the Center for 
Response to Intervention in Early Childhood; 
Charles Greenwood and Judith Carta, co-principal 
investigators; and Howard Goldstein, Scott McCo- 
nnell, and Ruth Kaminski, subcontracting investi- 
gators. Preparation of this manuscript was 
supported by Grant R324A170048 from the 
National Center for Special Education Research, 
Institute for Educational Science; Charles Green- 
wood, principal investigator; and the Kansas Intel- 
lectual and Developmental Disabilities Research 
Center (HD002528). The opinions presented in this 
paper are those of the authors, and no official 
endorsement from the Institute of Education Sci- 
ences should be inferred. 


Manuscript received April 2018; accepted October 
2018. 


Copyright of Exceptional Children is the property of Sage Publications Inc. and its content 
may not be copied or emailed to multiple sites or posted to a listserv without the copyright 
holder's express written permission. However, users may print, download, or email articles for 
individual use. 


